System and/or method for processing three dimensional images

ABSTRACT

The subject matter disclosed herein relates to a method and/or system for projection of images to appear to an observer as one or more three-dimensional images.

RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 61/033,169, filed on Mar. 3, 2008.

BACKGROUND

1. Field

The subject matter disclosed herein relates to processing images to beviewed by an observer.

2. Information

Three dimensional images may be created in theatre environments byilluminating a reflective screen using multiple projectors. For example,a different two-dimensional (2-D) image may be viewed in each of anobserver's eye to create an illusion of depth. Two-dimensional imagesgenerated in this manner, however, may result in distortion of portionsof the constructed three-dimensional image (3-D). This may, for example,introduce eye strain caused by parallax, particularly when viewing 3-Dimages generated over large areas.

BRIEF DESCRIPTION OF THE FIGURES

Non-limiting and non-exhaustive embodiments will be described withreference to the following figures, wherein like reference numeralsrefer to like parts throughout the various figures unless otherwisespecified.

FIG. 1 is a schematic a conventional system for projecting athree-dimensional (3-D) image to be viewed by an observer.

FIG. 2 is a schematic diagram illustrating effects of parallaxassociated with viewing a projected 3-D image.

FIGS. 3A through 3D are schematic diagrams of a system for projecting a3-D image over a curved surface according to an embodiment.

FIG. 4 is a schematic diagram of a system of capturing images of anobject for projection as a 3-D image according to an embodiment.

FIG. 5 is a schematic diagram of a system for generating compositeimages from pre-rendered image data and image data captured in real-timeaccording to an embodiment.

FIG. 6 is a schematic diagram of a 3-D imaging system implemented in atheater environment according to an embodiment.

FIG. 7 is a schematic diagram of a system for obtaining image databased, at least in part, on audience members sitting in a theateraccording to an embodiment.

FIG. 8 is a schematic diagram of a system for processing image dataaccording to an embodiment.

FIG. 9 is a diagram illustrating a process of detecting locations ofblobs based, at least in part, on video data according to an embodiment.

DETAILED DESCRIPTION

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of claimed subject matter. Thus, theappearances of the phrase “in one embodiment” or “an embodiment” invarious places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in one or moreembodiments.

According to an embodiment, an observer of a three-dimensional (3-D)image created from projection of multiple two-dimensional (2-D) imagesonto a reflective screen may experience parallax defined by one or moredeviation angles in at least some portions of the 3-D image. Suchparallax may be particularly acute as an observer views points on thereflective screen that are furthest from the center of the projected 3-Dimage. In one embodiment, multiple-projectors may project a 3-D image ona reflective screen from 2-D image data. Here, for example, a projectormay project an associated 2-D component of a 3-D image based, at leastin part, on digitally processed image data representative of a 2-Dimage.

As shown in FIG. 1, projectors 14 may each project a component of animage onto reflective screen 12 which may be perceived by an observer asone or more 3-D images of objects in a theater environment 10. Such 3-Dimages may comprise images of still or moving objects. In a theaterenvironment, such a 3-D image may be viewable through inexpensivepassive polarized glasses acting to interleave multiple 2-D images toappear as the 3-D image. Accordingly, two different views are projectedonto screen 12 where each of an observer's eyes sees its own view,creating an impression of depth. Here, such multiple 2-D images may beprojected with polarized light such that the reflected images are out ofphase by 90 degrees, for example. Alternatively, such a 3-D image may beviewable through active glasses which temporally interleave left andright components (e.g., at 120 Hz.) to appear as the 3-D image.

To create image data for use in projecting such 3-D images, multiplecameras may be positioned to capture an associated 2-D image of a 3-Dobject. Here, each camera may be used to capture image datarepresentative of an associated 2-D image of the object. Such 2-D imagescaptured by such cameras may then be processed and/or transformed into2-D components of a 3-D image to be projected by multiple projectorsonto a reflective screen as illustrated above.

In viewing a 3-D image generated as discussed above, an observer may seeviews of the same object that are not horizontally aligned, resulting inparallax. Here, such misalignment of views of the same object mayresult, at least in part, from placement of multiple projectors tocreate a 3-D image. For example, a 6.5 cm separation between anobserver's eyes may cause each eye to see a different image. Dependingon placement of projectors and separation of a viewer's eyes, such aviewer may experience parallax when viewing portions of a resulting 3-Dimage.

FIG. 2 shows a different aspect of theater environment 10 wherereflective screen 12 is flat or planar, and an observer obtainsdifferent views of a 3-D object at each eye 16 and 18. Here, eye 16obtains a view of a first 2-D image bounded by points 20 and 22 whileeye 18 obtains a view of a second 2-D image bounded by points 24 and 26.As shown in FIG. 2, such first and second images are horizontallynon-aligned and/or skewed on the flat or planar reflective screen 12 asviewed by respective eyes 16 and 18. Accordingly, the observer mayexperience parallax and/or eye strain.

Briefly, one embodiment relates to a system and/or method of generatinga 3-D image of an object. 2-D images of a 3-D object may be representedas 2-D digital image data. 2-D images generated from such 2-D image datamay be perceived as a 3-D image by an observer viewing the 2-D images.At least a portion of the 2-D image data may be transformed forprojection of associated 2-D images onto a curved surface by, forexample, skewing at least a portion of the digital image data. Suchskewing of the digital image data may reduce a deviation errorassociated with viewing a projection of a resulting 3-D image by anobserver. It should be understood, however, that this is merely anexample embodiment and claimed subject matter is not limited in thisrespect.

FIGS. 3A through 3D show views of a system 100 for generating 3-D imagesviewable by an observer facing reflective screen 112. Projectors 114project 2-D images onto reflective screen 112. The combination of thereflected 2-D images may appear to the observer as a 3-D image. Here, itshould be observed that reflective screen 112 is curved along at leastone dimension. In this particular embodiment, reflective screen 112 iscurved along an axis that is vertical with respect to an observer'ssight while projectors 114 are positioned to project associatedoverlapping images of an object onto reflective screen 112. Further, inthe particularly illustrated embodiment, projectors 114 are positionedat height to project images downward and over the heads of observers(not shown). Accordingly, 3-D images may be made to appear in front ofsuch observers, and at below eye level. In a particular embodiment,multiple projectors may be placed such that optical axes of the lensintersect roughly at a single point on a reflective screen at aboutwhere an observer is to view a 3-D image. In some embodiments, multiplepairs of projectors may be used for projecting multiple 3-D images overa panoramic scene, where each pair of projectors is to project anassociated 3-D image in the scene. Here, for example, each projector insuch a projector pair may be positioned such that the optical axes ofthe lenses in the projector pair intersect at a point on a reflectivescreen.

As illustrated below, by processing 2-D image data for projection ontosuch a curved surface, distortions in the resulting 3-D (as perceived bythe observer) may be reduced. As referred to herein, a “curved”structure, such as a reflective screen and/or surface, comprises asubstantially non-planer structure. Such a curved screen and/or surfacemay comprise a smooth surface contour with no abrupt changes indirection. In the particular embodiment illustrated in FIGS. 3A through3D, reflective screen 12 may be formed as a curved screen comprising aportion of a circular cylinder having reflective properties on a concavesurface. Such a cylindrical curved screen may have any radius ofcurvature such as, for example, four feet or smaller, or larger thanthirteen feet. In other embodiments, however, a curved screen maycomprise curvatures of different geometrical shapes such as, forexample, spherical surfaces, sphereoidal surfaces, parabolic surfaces,hyperbolic services or ellipsoidal surfaces, just to name a fewexamples.

According to an embodiment, projectors 114 may transmit polarized images(e.g., linearly or circularly polarized images) that are 90° out ofphase from one another. Accordingly, an observer may obtain an illusionof depth by constructing a 3-D image through glasses having left andright lenses polarized to match associated reflected images. As shown inthe particular embodiment of FIGS. 3A through 3D, portions of 2-D imagesprojected onto screen 112 may partially overlap. Here, screen 112 maycomprise a gain screen or silver screen having a gain in a range ofabout 1.8 to 2.1 to reduce or inhibit the intensity of “hot spots”viewed by an observer in such regions where 2-D images overlap, and topromote blending of 2-D images while maintaining polarization

As pointed out above, images viewed through the left and right eyes ofan observer (constructing a 3-D image) may be horizontally skewed withrespect to one another due to parallax. According to an embodiment,although claimed subject matter is not so limited, image data to be usedin projecting an image onto a curved screen (such as screen 112) may beprocessed to reduce the effects of parallax and horizontal skewing.Here, projectors 114 may project images based, at least in part, ondigital image data representative of 2-D images. In a particularembodiment, such digital image data may be transformed for projection ofmultiple images onto a curved surface appearing to an observer as a 3-Dimage as illustrated above. Further, such digital image data may betransformed for horizontal de-skewing of at least a portion of theprojection of the multiple images as viewed by the observer.

FIG. 4 is a schematic diagram of a system 200 for capturing 2-D imagesof a 3-D object 202 for use in generating a 3-D image 254. Here,multiple cameras 214 may obtain multiple 2-D images of 3-D object 202 atdifferent angles as shown. Such cameras may comprise any one of severalcommercially available cameras capable of digitally capturing 2-D imagessuch as high definition cameras sold by Sony, for example. However, lessexpensive cameras capable of capturing 2-D images may also be used, andclaimed subject matter is not limited to the use of any particular typeof camera for capturing images.

Digital image data captured at cameras 214 may be processed at computingplatform 216 to, among other things generate digital image datarepresenting images to be projected by projectors 220 against a curvedreflective screen 212 for the generation of 3-D image 254. In thepresently illustrated embodiment, such 2-D images are represented asdigital image data in a format such as, for example, color bit-map pixeldata including 8-bit RGB encoded pixel data. However, other formats maybe used without deviating from claimed subject matter.

Cameras 214 may be positioned to uniformly cover portions of interest ofobject 202. Here, for example, cameras 214 may be evenly spaced toevenly cover portions of object 202. In some embodiments, a higherconcentration of cameras may be directed to portions of object 202having finer details and/or variations to be captured and projected as a3-D image. Projectors 220 may be placed to project 2-D images ontoscreen 212 to be constructed by a viewer as 3-D image 254 as illustratedabove. Also, and as illustrated above with reference to FIGS. 3A through3D, projectors 220 may be positioned so as to not obstruct an observer'sview of images on screen 212 viewers in an audience. For example,projectors 220 may be placed over head, at foot level and/or to the sideof an audience that is viewing 3-D image 254.

According to an embodiment, cameras 214 may be positioned with respectto object 202 independently of the positions of projectors 220 withrespect to screen 212. Accordingly, based upon such positioning ofcameras 214 and projectors 220, a warp engine 218 may transform digitalimage data provided by computing platform 216 relative to placement ofprojectors 220 to account for positioning of cameras 214 relative toprojectors 220. Here, warp engine 218 may employ one or more affinetransformations using techniques known to those of ordinary skill in theart. Such techniques applied to real-time image warping may includetechniques described in King, D. Southcon/96. Conference Record Volume,Issue 25-27, June 1996, pp. 298-302.

According to an embodiment, computing platform 216 and/or warp engine218 may comprise combinations of computing hardware including, forexample, microprocessors, random access memory (RAM), mass storagedevices (e.g., magnetic disk drives or optical memory devices),peripheral ports (e.g., for communicating with cameras and/orprojectors) and/or the like. Additionally, computing platform 216 andwarp engine may comprise software and/or firmware enablingtransformation and/or manipulation of digital image data captured atcameras 214 for transmitting images onto screen 212 through projectors220. Additionally, while warp engine 218 and computing platform 216 areshown as separate devices in the currently illustrated embodiment, itshould be understood that in alternative implementations warp engine 218may be integrated with computing platform 216 in a single device and/orcomputing platform.

According to an embodiment, images of object 202 captured at cameras 214may comprise associated 2-D images formed according to a projection offeatures of object 202 onto image planes associated with cameras 214.Accordingly, digital image data captured at cameras 214 may comprisepixel values associated with X-Y positions on associated image planes.In one particular implementation, as illustrated above, images projectedon to a reflective screen, and originating at different cameras, may behorizontally skewed as viewed by the eyes of an observer. As such,computing platform 216 may process such 2-D image data captured atcameras 214 for projection on to the curvature of screen 212 by, forexample, horizontally de-skewing at least a portion of the 2-D imagedata, thereby horizontally aligning images originating at differentcameras 214 to reduce parallax experienced by an observer viewing aresulting 3-D image.

According to an embodiment, a location of a feature of object 202 on animage plane of a particular camera 214 may be represented in Cartesiancoordinates x and y which are centered about an optical axis of theparticular camera 214. In one particular implementation, and withoutadjusting for horizontal skew of images, such a location may bedetermined as follows:

${\lambda \begin{bmatrix}x \\y \\1\end{bmatrix}} = {\begin{bmatrix}{- f} & 0 & 0 & 0 \\0 & {- f} & 0 & 0 \\0 & 0 & 1 & 0\end{bmatrix}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}$

Where:

-   -   X, Y and Z represent a location of an image feature on object        202 in Cartesian coordinates having an origin located on an        image plane of the particular camera 214, and where dimension Z        is along its optical axis;    -   x and y represent a location of the image feature in the image        plane;    -   ƒ is a focal length of the particular camera 214; and    -   λ is a non-zero scale factor.

According to an embodiment, an additional transformation may be appliedto 2-D image data captured at a camera 214 (e.g., at computing platform216) to horizontally de-skew a resulting 2-D image as projected ontoreflective screen 212 with respect to one or more other 2-D imagesprojected onto reflective screen 212 (e.g., to reduce the incidence ofparallax as viewed by the observer). Here, such a transformation may beexpressed as follows:

$\begin{bmatrix}x^{\prime} \\y^{\prime} \\1\end{bmatrix} = {\begin{bmatrix}1 & 0 & u_{0} \\0 & 1 & v_{0} \\0 & 0 & 1\end{bmatrix} \times \begin{bmatrix}x \\y \\1\end{bmatrix}}$

Where:

x′ and y′ represent a transformed location of the image feature in theimage plane;

u₀ represents an amount that a location is shifted horizontally; and

v₀ represents an amount that a location is shifted vertically.

Here, the value u₀, affecting the value x′, may be selected tohorizontally de-skew a resulting projected image from one or more otherimages viewed by an observer from a reflective screen as discussedabove. As pointed out above, projectors may be positioned such thatoptical axes intersect at a point on a reflected screen to reconstructtwo 2-D images as a 3-D image. By adjusting the value of u₀, aneffective or virtual optical axis of a 2-D image may be horizontallyshifted to properly align 2-D images projected by two differentprojectors. For example, values of u₀ for images projected by a pair ofprojectors may be selected such that resulting images projected by theprojectors align at a point on a reflective screen at a center betweenthe pair of projectors. While there may be a desire to de-skew imageshorizontally (e.g., in the direction of x) in a particular embodiment,there may be no desire to de-skew images vertically (e.g., in thedirection of y). Accordingly, the value v₀ may be set at zero. Values ofu₀ may be determined based on an analysis of similar triangles that areset by the focal length based upon a location of the observer relativeto the screen.

System 200 may be used to project still or moving images of objects ontoscreen 212 for viewing by an observer as a 3-D image. In one particularembodiment, as illustrated in FIG. 5, real-time images of objects may beprojected onto a screen to appear as 3-D images to an observer where atleast one portion of the projected image is based upon an image of anobject captured in real-time. Here, system 300 may project images onto ascreen based, at least in part, on digital image data generated by apre-render system 304 and generated by real-time imaging system 306.

Projectors 316 may project 2-D images onto a reflective screen (e.g., acurved screen as illustrated above) to be perceived as 3-D images to anobserver. In the particularly illustrated embodiment, sequentialconverters 314 may temporally interleave right and left 2-D images. Inan alternative implementation, projectors 316 may transmit left andright 2-D images that are polarized and 90° out of phase, permitting anobserver wearing eye glasses with polarized lens to view associated leftand right components to achieve the illusion of depth as illustratedabove.

According to an embodiment, portions of images generated by pre-rendersystem 304 and generated by real-time imaging system 306 may bedigitally combined at an associated compositor 312. Real-time computergenerated imagery (CGI) CPUs 310 are adapted process digital image dataof images of objects captured at one or more external cameras 320 incamera system 318. For example, real-time CGI CPUs 310 may comprisecomputing platforms adapted to process and/or transform images ofobjects using one or more techniques as illustrated above (e.g., toreduce parallax as experienced by an observer). In one embodiment, theone or more external cameras 320 may controlled (e.g., focus, pointing,zoom, exposure time) automatically in response to signals received attracking system 322. Here, tracking system 322 may include sensors suchas, for example, IR detectors, microphones, vibration sensors and/or thelike to detect the presence and/or movement of objects which are to beimaged by the one or more external cameras 320. Alternatively, or inconjunction with control from tracking system 322, cameras 322 may becontrolled in response to control signals from external camera control302.

According to an embodiment, pre-render system 304 comprises one or morevideo servers 308 which are capable of generating digital video imagesincluding, for example, images of scenes, background, an environment,animated characters, animals, actors and/or the like, to be combinedwith images of objects captured at camera 302. Accordingly, such imagesgenerated by video servers 308 may complement images of objects capturedat camera system 318 in a combined 3-D image viewed by an observer.

According to a particular embodiment, system 200 may be implemented in atheatre environment to provide 3-D images to be viewed by an audience.For example, system 400 shown in FIG. 6 is adapted to provide 3-D imagesfor viewing by audience members 426 arranged in an amphitheater seatingarrangement as shown. As illustrated above according to particularembodiments, projectors 420 may be adapted to project 2-D images ontocurved reflective screen 412 to be viewed as 3-D images by audiencemembers 426. Such 2-D images may be generated based, at least in part,on combinations image data provided by pre-render systems 404 andreal-time digital image data generated from capture of images of anobject by cameras 414, for example. As illustrated above in FIG. 5according to a particular embodiment, compositors 424 may digitallycombine 2-D images processed by associated computing platforms 404 withpre-rendered image data from associated pre-render systems 404.

According to an embodiment, cameras 414 may be placed in a location soas to not obstruct the view of audience members 426 in viewing 3-D image454. For example, cameras 414 may be placed above or below audiencemembers 426 to obtain a facial view. Similarly, projectors may bepositioned overhead to project downward onto curved screen 412 to createthe appearance of 3-D image 454.

Digital image data captured at a camera 414 may be processed at anassociated computing platform 416 to, for example, reduce parallax asexperienced by audience members 426 in viewing multiple 2-D images as asingle 3-D image using one or more techniques discussed above.Additionally, combined image data from a combiner 424 may be furtherprocessed by an associated warp engine to, for example, account forpositioning of a projector 420 relative to an associated camera 414 forgenerating a 2-D image to appear to audience members 426, along withother 2-D images, as a 3-D image 454.

In one implementation, cameras 414 may be controlled to capture an imageof a particular audience member 428 for generating a 3-D image 454 to beviewed by the remaining audience members 426. As illustrated above,cameras 414 may be pointed using, for example, an automatic trackingsystem and/or manual controls to capture an image of a selected audiencemember. Here, horizontal de-skewing of 2-D images may be adjusted basedon placement of cameras 414 relative to the location of such a selectedaudience member. For example, parameters linear transformations (such asu₀ discussed above) applied to 2-D image data may respective projectionmatrices. Pre-rendered image data from associated pre-render systems 404may be combined with an image of audience member 428 to provide acomposite 3-D image 454. Such pre-rendered image data may provide, forexample, outdoor scenery, background, a room environment, animatedcharacters, images of real persons and/or the like. Accordingly,pre-rendered image data combined at combiners 424 may generateadditional imagery appearing to be co-located with the image of audiencemember 428 in 3-D image 454. Such additional imagery appearing to beco-located with the image of audience member 428 in 3-D image 454 mayinclude, for example, animated characters and/or people interacting withaudience member 428. In addition, system 400 may also generate soundthrough an audio system (not shown) that is synchronized with thepre-rendered image data for added effect (e.g., voice of individual oranimated character that is interacting with an image of audience member428 recast in 3-D image 454).

According to an embodiment, system 400 may include additional cameras(not shown) to detect motion of audience members 426. Such cameras maybe located, for example, directly over audience members 426. In oneparticular implementation, such over head cameras may include aninfrared (IR) video camera such as IR video camera 506 shown in FIG. 7.Here, audience members (not shown) may generate and/or reflect energydetectable at IR video camera 506. In one embodiment, an audience membermay be lit by one or more IR illuminators 505 and/or otherelectromagnetic energy source capable of generating electromagneticenergy with a relatively limited wavelength range.

IR illuminators 505 may employ multiple infrared LEDs to provide abright, even field of infrared illumination over area 504 such as, forexample, the IRL585A from Rainbow CCTV. IR Camera 506 may comprise acommercially available black and white CCD video surveillance camerawith any internal infrared blocking filter removed or other video cameracapable of detection of electromagnetic energy in the infraredwavelengths. IR pass filter 508 may be inserted into the optical path ofcamera 506 optical path to sensitize camera 506 to wavelengths emittedby IR illuminator 505, and reduce sensitivity to other wavelengths. Itshould be understood that, although other means of detection arepossible without deviating from claimed subject matter, human eyes areinsensitive to infrared illumination and such infrared illumination maynot interfere with visible light in interactive area 504 or alter a moodin a low-light environment.

According to an embodiment, information collected from images of one ormore audience members captured at IR camera 506 may be processed in asystem as illustrated according to FIG. 8. Here, such information may beprocessed to deduce one or more attributes or features of individualsincluding, for example, motion, hand gestures, facial expressions and/orthe like. In this particular embodiment, computing platform 620 isadapted to detect X-Y positions of shapes or “blobs” that may be used,for example in determining locations of audience members (e.g., audiencemembers 426), facial features, eye location, hand gestures, presence ofadditional individuals co-located with individuals, posture and positionof head, just to name a few examples. Also, it should be understood thatspecific image processing techniques described herein are merelyexamples of how information may be extracted from raw image data indetermining attributes of individuals, and that other and/or additionalimage processing techniques may be employed.

According to an embodiment, positions of one or more audience membersmay be associated with one or more detection zones. Using informationobtained from overhead cameras such as IR camera 506, movement of anindividual audience member 426 may be detected by monitoring detectionzones for each position associated with an audience member 426. As such,cameras 414 may be controlled to capture images of individuals inresponse to detection of movement of individuals such as, for example,hand gestures. Accordingly, audience members 426 may interact with videocontent (e.g., from image data provided by pre-render systems 404)and/or interactive elements.

In one particular example, detection of gestures from an audience membermay be received as a selection of a choice or option. For example, suchdetection of a gesture may be interpreted as a vote, answer to amultiple choice question, selection of a food or beverage to be orderedand brought to the audience member's seat and/or the like. In anotherembodiment, such gestures may be interpreted as request to changepresentation, brightness, sound level, environmental controls (e.g.,heating and air conditioning) and/or the like.

According to an embodiment, information from IR camera 506 may bepre-processed by circuit 610 to compare incoming video 601 signal fromIR camera 506, a frame at a time, against a stored video frame 602captured by IR camera 506. Stored video frame 602 may be captured whenare 504 is devoid of individuals or other objects, for example. However,it should be apparent to those skilled in the art that stored videoframe 602 may be periodically refreshed to account for changes in anenvironment such as area 504.

Video subtractor 603 may generate difference video signal 608 by, forexample, subtracting stored video frame 602 from the current frame. Inone embodiment, this difference video signal may display onlyindividuals and other objects that have entered or moved within area 504from the time stored video frame 602 was captured. In one embodiment,difference video signal 608 may be applied to a PC-mounted videodigitizer 621 which may comprise a commercially available digitizingunit, such as, for example, the PC-Vision video frame grabber fromCoreco Imaging.

Although video subtractor 610 may simplify removal of artifacts within afield of view of camera 506, a video subtractor need not be necessary.By way of example, without intending to limit claimed subject matter,locations of targets may be monitored over time, and the system mayignore targets which do not move after a given period of time until theyare in motion again.

According to an embodiment, blob detection software 622 may operate ondigitized image data received from A/D converter 621 to, for example,calculate X and Y positions of centers of bright objects, or “blob”, inthe image. Blob detection software 622 may also calculate the size ofsuch detected blob. Blob detection software 622 may be implemented usinguser-selectable parameters, including, but not limited to, low and highpixel brightness thresholds, low and high blob size thresholds, andsearch granularity. Once size and position of any blobs in a given videoframe are determined, this information may be passed to applicationssoftware 623 to determine deduce attributes of one or more individuals503 in area 504.

FIG. 8 depicts a pre-processed video image 608 as it is presented toblob detection software 622 according to a particular embodiment. Asdescribed above, blob detection software 622 may detect individualbright spots 701, 702, 703 in difference signal 708, and the X-Yposition of the centers 710 of these “blobs” is determined. In analternative embodiment, the blobs may be identified directly from thefeed from IR camera 506. Blob detection may be accomplished for groupsof contiguous bright pixels in an individual frame of incoming video,although it should be apparent to one skilled in the art that the framerate may be varied, or that some frames may be dropped, withoutdeparting from claimed subject matter.

As described above, blobs may be detected using adjustable pixelbrightness thresholds. Here, a frame may be scanned beginning with anoriginating pixel. A pixel may be first evaluated to identify thosepixels of interest, e.g. those that fall within the lower and upperbrightness thresholds. If a pixel under examination has a brightnesslevel below the lower brightness threshold or above the upper brightnessthreshold, that pixel's brightness value may be set to zero (e.g.,black). Although both upper and lower brightness values may be used forthreshold purposes, it should be apparent to one skilled in the art thata single threshold value may also be used for comparison purposes, withthe brightness value of all pixels whose brightness values are below thethreshold value being reset to zero.

Once the pixels of interest have been identified, and the remainingpixels zeroed out, the blob detection software begins scanning the framefor blobs. A scanning process may begin with an originating pixel. Ifthat pixel's brightness value is zero, a subsequent pixel in the samerow may be examined. A distance between the current and subsequent pixelis determined by a user-adjustable granularity setting. Lowergranularity allows for detection of smaller blobs, while highergranularity permits faster processing. When the end of a given row isreached, examination proceeds with a subsequent row, with the distancebetween the rows also configured by the user-adjustable granularitysetting.

If a pixel being examined has a non-zero brightness value, blobprocessing software 622 may begin moving up the frame-one row at a timein that same column until the top edge of the blob is found (e.g., untila zero brightness value pixel is encountered). The coordinates of thetop edge may be saved for future reference. Blob processing software 622may then return to the pixel under examination and moves down the rowuntil the bottom edge of the blob is found, and the coordinates of thebottom edge are also saved for reference. A length of the line betweenthe top and bottom blob edges is calculated, and the mid-point of thatline is determined. A mid-point of the line connecting the detected topand bottom blob edges then becomes the pixel under examination, and blobprocessing software 622 may locate left and right edges through aprocess similar to that used to determine the top and bottom edge. Themid-point of the line connecting the left and right blob edges may thenbe determined, and this mid-point may become the pixel underexamination. Top and bottom blob edges may then be calculated againbased on a location of the new pixel under examination. Once approximateblob boundaries have been determined, this information may be stored forlater use. Pixels within the bounding box described by top, bottom,left, and right edges may then be assigned a brightness value of zero,and blob processing software 622 begins again, with the original pixelunder examination as the origin.

Although this detection software works well for quickly identifyingcontiguous bright regions of uniform shape within the frame, thedetection process may result in detection of several blobs where onlyone blob actually exists. To remedy this, blob coordinates may becompared, and any blobs intersecting or touch may be combined togetherinto a single blob whose dimensions are the bounding box surrounding theindividual blobs. The center of a combined blob may also be computedbased, at least in part, on the intersection of lines extending fromeach corner to the diagonally opposite corner. Through this process, adetected blob list, which may include, but not be limited to including,the center of blob; coordinates representing the blob's edges; a radius,calculated as a mean of the distances from the center of each of theedges for example; and the weight of a blob, calculated as a percentageof pixels within the bounding rectangle which have a non-zero value forexample, can be readily determined.

Thresholds may also be set for the smallest and largest group ofcontiguous pixels to be identified as blobs by blob processing software622. By way of example, without intending to limit claimed subjectmatter, where a uniform target size is used and the size of theinteraction area and the height of the camera above area 504 are known,a range of valid target sizes can be determined, and any blobs fallingoutside the valid target size range can be ignored by blob processingsoftware 622. This allows blob processing software 622 to ignoreextraneous noise within the interaction area and, if targets are used,to differentiate between actual targets in the interaction area andother reflections, such as, but not limited to, those from anyextraneous, unavoidable, interfering light or from reflective clothingworn by an individual 503, as has become common on some athletic shoes.Blobs detected by blob processing software 622 falling outside thresholdboundaries set by the user may be dropped from the detected blob list.

Although one embodiment of computer 620 of FIG. 8 may include both blobprocessing software 622 and application logic 623, blob processingsoftware 622 and application logic 623 may be constructed from a modularcode base allowing blob processing software 622 to operate on onecomputing platform, with the results therefrom relayed to applicationlogic 623 running on one or more other computing platforms.

While there has been illustrated and described what are presentlyconsidered to be example embodiments, it will be understood by thoseskilled in the art that various other modifications may be made, andequivalents may be substituted, without departing from claimed subjectmatter. Additionally, many modifications may be made to adapt aparticular situation to the teachings of claimed subject matter withoutdeparting from the central concept described herein. Therefore, it isintended that claimed subject matter not be limited to the particularembodiments disclosed, but that such claimed subject matter may alsoinclude all embodiments falling within the scope of the appended claims,and equivalents thereof.

I claim:
 1. A method comprising: projecting one or more images in a theater; detecting a gesture from an audience member in the theater, wherein the gesture interacts with the one or more projected images; and interpreting the gesture.
 2. The method of claim 1, wherein the detection is performed by capturing images of the audience member in response to detection of movement of the audience member.
 3. The method of claim 1, wherein the detection is performed by capturing an infrared illumination of the audience member.
 4. The method of claim 1, wherein the interpretation of the gesture is performed through blob processing of one or more attributes of the audience member.
 5. The method of claim 4, wherein the one or more attributes of the audience member is selected from the group consisting of facial features, location of eyes, location of hands, and head positioning.
 6. A system comprising: a projector that projects one or more images in a theater; a detection device that detects a gesture from an audience member in the theater, wherein the gesture interacts with the one or more projected images; and a processor that interprets the gesture.
 7. The system of claim 6, wherein the detection device is an infrared camera that captures images of the audience member in response to detection of movement of the audience member.
 8. The system of claim 6, further comprising an infrared illuminator that illuminates the audience member such that the detection device detects the gesture.
 9. The system of claim 6, further comprising a processor that interprets the gesture through blob processing of one or more attributes of the audience member.
 10. The system of claim 9, wherein the one or more attributes of the audience member is selected from the group consisting of facial features, location of eyes, location of hands, and head positioning.
 11. An apparatus comprising: a detection device that detects a gesture from an audience member in a theater and provides the gesture to a processor for interpretation, wherein the gesture interacts with one or more images projected by a projector in the theater. 